Modelling human perception to leverage the reuse of concepts across the multi-sensory design space

Information Visualisation is an emerging discipline that concerns the design of interactive computer systems that provide the user with a visual model of abstract data. Information Visualisation implies a mapping from the data attributes to the units of visual perception. Information Sonification is an embryonic field that uses sound rather than imagery to present abstract data. Information Sonification, implies a mapping from the data attributes to the units of auditory perception. In both these fields the need to describe appropriate mappings between the data and the units of perception has led to models or taxonomies that describe the available design space. While these models of the visual design space and the auditory design space may be appropriate for people working in a single sensory domain, these models based purely on sensory attributes are very disjoint. However, for designers who wish to consider a multi-sensory solution to information display, these disjoint models of the different sensory domains make it difficult to compare and contrast the possible mapping choices. This paper describes existing conceptual models of the visual and auditory design space and then proposes a different conceptual modelling of the multi-sensory design space. This new model describes the units of perception but is not based on sensory attributes, but typical information metaphors. Throughout the paper all discussions are illustrated using the UML modelling notation which is a standard notation used to document the design of software systems.